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Abstract 

This paper presents the results of a study 
that was conducted to identify and document 
student perceptions of the effectiveness of com- 
puter modeling software introduced in a design 
foundations course that had previously utilized 
only conventional manually-produced represen- 
tation techniques. Rather than attempt to utilize 
a production-oriented CAD program, this 
exploratory study investigated the use of alterna- 
tive software applications developed by vendors 
specifically to function as conceptual design 
tools. The study was structured to identify areas 
of future investigation related to student percep- 
tions of the effectiveness of the software in 
facilitating design exploration, the effectiveness 
and ease of use of various application features, 
and the extent to which prior CAD coursework 
was related to the perceived effectiveness of the 
software on design outcomes. 

Introduction 

The pervasiveness of computing in architec- 
tural practice poses challenges for design educa- 
tors as faculty attempt to address the demands 
from practicing professionals for graduates who 
have greater competencies with digital technolo- 
gies (Mitgang, 1999). The role of computing in 
architectural practice has evolved to CAD’s 
becoming a baseline requirement for clients 
(Green, 2002). However, the emphasis of design 
education on developing design skills and pro- 
moting conceptual exploration has given rise to 
an ongoing debate among educators about the 
role of CAD and related digital tools in design 
curriculums. Further, it has been argued that the 
mandate of architectural education is to cultivate 
skills in life-long learning and that practice is 
where students develop technical knowledge 
(Karloff, 1996). 

For educators attempting to integrate digital 
media in design courses, the criteria for select- 
ing appropriate applications and tools for meet- 
ing course objectives is primarily subjective. 
More important, the literature on this topic com- 
monly focuses on the success of students when 
using digital applications in design projects 
from the instructor’s perspective, rather than 


from the novice designer’s point of view. 
Therefore, research conducted to identify the 
extent to which students found software effec- 
tive in the design process can provide insight 
into how digital design tools are best utilized by 
novice designers and assist design faculty in 
selecting appropriate digital applications to sup- 
port intended course outcomes . In order to 
develop a basis for making such decisions, a 
study was conducted to begin to establish a 
knowledge base that could help identify and 
document student perceptions of the effective- 
ness of a software application. 

To explore the use of an alternative to pro- 
duction-oriented CAD programs, this study was 
structured to investigate software applications 
developed specifically to function as conceptual 
design tools in introductory design courses. The 
study focused on student’s perception related to 
the software effectiveness in facilitating design 
exploration, the effectiveness and ease of use of 
various application features, and the extent to 
which prior CAD skills were related to the per- 
ceived effectiveness of the software on design 
outcomes. Given the emphasis of these applica- 
tions on ease of use, issues related to the influ- 
ence of prior CAD skills on students perceptions 
were of particular interest. 

All participants in the study were architec- 
ture majors enrolled in the second class of a 
two-course sequence covering concepts related 
to architectural representation and design foun- 
dations. Activities associated with the study took 
place during the final project of the course. The 
students were provided a project brief; they were 
required to use manual drawing and physical 
modeling in parallel with computer modeling 
using software developed for conceptual design 
in the development of their individual design 
solutions. Data was gathered using an instructor- 
developed survey administered at the end of the 
course. Instructor observation and statistical 
analysis of the survey data were used to develop 
an initial assessment of the perceived effective- 
ness of the software among students in the 
course. 
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Literature Review 

Conceptual Design Processes and the 
Limitations of CAD 

Sketching has traditionally been an integral 
activity in conceptual design. The importance of 
the sketch in the design process has been attrib- 
uted to both its role in the iterative actions asso- 
ciated with conceptual design as well as to the 
contribution of ambiguity. Laseau (2000) dis- 
cussed the importance of sketching as a means 
of facilitating “graphic thinking” in the design 
process, stating that graphic thinking facilitated 
a “communication loop” between the paper, the 
eye, the hand, and the brain, and that the poten- 
tial of graphic thinking lies in the continuous 
cycling of information (p. 8). Goel (1995) 
referred to the sketch as a non-notational sym- 
bol system that promotes cognitive shifts from 
one proposed conceptual idea to other alternative 
concepts, a process he referred to as lateral 
transformation. He stated that “the general claim 
is that lateral transformations need to occur 
during the preliminary phase of design problem 
solving and that the density and ambiguity of the 
symbol system of sketching facilitate these cogni- 
tive operations” (p. 194) and cited the role of 
ambiguity in promoting lateral transformations or 
shifts between alternative concepts and solutions. 

The visually ambiguous features of a sketch 
have been associated with the cognitive process- 
es needed for design exploration. Won (2001) 
argued that during the drawing process designers 
demonstrate a “seeing behavior,” in which they 
will concentrate on the figural properties of a , 
thus enabling the designer to “see the image as 
something else” (p. 324). Similarly, Suwa and 
Tversky (1997) proposed that sketches enable 
designers to “see unanticipated relations and fea- 
tures that suggest ways to refine and revise 
ideas” (p. 386). 

In architectural design, CAD (computer- 
aided drafting or design) has become a primary 
medium for the development of construction 
drawings in professional practice. However, the 
inherent emphasis of CAD on precision has 
“accentuated the divide between explicit and 
implicit information” (Johnson, 1998). The pen 
or pencil sketch has retained its prominence as a 
design tool because it is exploratory rather than 
absolute, because it is inherently ambiguous, and 
because the degree to which the information it 
conveys is implied and subjective. Chastain, 
Kalay, and Peri (2002) argued that CAD is inher- 


ently unambiguous because its representations 
are explicit rather than implicit and therefore it is 
inappropriate for early stages of the design in 
which much of the effort is focused on conceptu- 
al characteristics. 

The lack of ambiguity in computer-generat- 
ed representations has lead to the view that com- 
puter applications are ineffective as design tools 
in the early stages of the design process where 
the priority is creativity (Van Elsas & Veergeest, 
1998; Leglise, 1995; Suwa & Tvurskey, 1997). 
Jonson (2002) proposed that the strength of the 
freehand sketch lies in its economy of means 
(low cost) and immediacy (single-tool interface). 
He argued that a sketch carries less redundancy 
than a final drawing, stating that “when a sketch 
represents more than one possible interpretation, 
it could be seen as an explorative tool” (p. 248). 
This position directly contrasts with the purpose 
of a drafted image which has the intent of pro- 
moting the acceptance of a specific 
interpretationOutput generated with CAD appli- 
cations is similarly viewed as lacking the ambi- 
guity necessary to effectively promote the multi- 
ple interpretations and visual shifts in perception 
associated with design exploration in design 
processes (Won, 2001). 

According to Bilda and Demirken (2003), 
CAD software works primarily on a “draw then 
modify” principle. From their research, they con- 
cluded that designers were more effective in 
using time, conceiving the problem, producing 
alternate solutions, and perceiving visual-spatial 
features and the organizational relations of a 
design in traditional rather than digital media 
during conceptual design. Other researchers have 
concluded that these limitations extend to other 
disciplines. In a study to investigate the compari- 
son between traditional media and digital media 
among novice graphic designers. Stones and 
Cassidy (2007) concluded that working on a 
computer “seems not to be as effective, both in 
terms of fluency and variety, as drawing the form 
on paper” (p. 70). 

The limitations of CAD in conceptual 
design also have been linked to the interface 
used. Designers typically use a mouse, a key- 
board, and a 2D screen to interact with CAD sys- 
tems. According to Ye, Campbell, Page, and 
Badni (2006), “the need to design and deploy 
new computer interfaces for the CAD system is 
evident, especially in support of conceptual 
design” (p.78). 



3D Modeling and Digital Sketching Tools 

According to Abdelhameed (2004), explor- 
ing design ideas through either two-dimensional 
or three-dimensional forms is the basis of design 
exploration and visual thinking during the 
design process. Whereas traditional media relied 
largely on sketching, 2D representations, and 
physical models for design exploration, comput- 
er-modeling tools have greatly expanded the 
ability of designers to develop 2D and 3D repre- 
sentations. For example, Simondetti (2002) 
investigated the implications of introducing 
computer-generated physical modeling in the 
early stages of the design process. He studied a 
design process that integrated sketching, com- 
puter modeling, and computer-generated physi- 
cal modeling. According to Simondetti, this 
recursive process provided the following advan- 
tages: (1) understanding kinetic design, (2) 
understanding design involving complex geome- 
try, and (3) understanding design at the interface 
with the human body. 

Re-conceptualizing the Design Process to 
Optimize Digital Tools 

Some researchers have suggested that the 
use of digital tools in conceptual design involves 
re-conceptualizing the design process itself in a 
way in which design with digital tools is per- 
ceived as involving an inherently different 
process. Abdelhameed (2004) argued that com- 
puting has “changed from being just a tool for 
drawing to being a medium through and by 
design which design is performed and solutions 
are generated” (p. 485). Bilda and Demirken 
(2003) proposed that “while digital media seems 
inconvenient for the conceptual design phase, 
this situation depends on the designer’s design- 
ing habits” as well as limitations of the software 
(p. 49). Therefore, the argument that digital 
design tools are inappropriate for conceptual 
design may be reframed as an argument that the 
traditional design processes are mismatched to 
the digital media environment. 

This re-conceptualization is reflected in 
arguments that propose that digital technologies 
be conceived as a design medium rather than a 
design tool. Oxman (2006) proposed that “inter- 
action with computational design media requires 
of the designer a different form of input and 
level of formalization” and that “these distinc- 
tions between paper-based interaction with rep- 
resentations and digital interactions are signifi- 
cant both cognitively and theoretically” (p. 243). 
Sequin (2001) stated that the computer enabled 


artists to “to tackle structures of a level of com- 
plexity that clearly exceeded what an unassisted 
human could hope to achieve in the conceptual 
design phase as well as in the actual implemen- 
tation of the final shape” (p. 345). He proposed 
that there was a “general trend to make comput- 
er based environments not simply a (still quite 
imperfect) emulation of real physical artist’s 
tools, but to exploit some unique and novel serv- 
ices that only a computer can offer” (p. 347). 

Conceptual Digital Design Environments 

Conceptual design processes differ from 
other design phases such as design development 
and production, and typical digital design appli- 
cations do not provide adequate support for all 
phases (Schodeck, Becthold, Griggs, Koa, & 
Steinberg, 2005). Effective conceptual design in 
a digital environment must rely on quick feed- 
back from (digital) sketches, 3D modeling, and 
visualization. Conceptual modeling environ- 
ments developed specifically for early design 
phases are “stand alone” sketching programs, 
geometric modelers, and renderers with an 
emphasis on initial shape generation that 
“employ line elements to facilitate the transition 
from traditional hand sketching to digital model- 
ing” and often utilize interactive tablets for input 
in a manner similar to sketching on paper 
(Schodeck, Becthold, Griggs, Koa, & Steinberg, 
2005, p. 193).According to Schodeck, Becthold, 
Griggs, Koa, & Steinberg (2005), “these envi- 
ronments must be simple to learn, easy to use, 
and not impose limits” (p. 192), This approach 
is consistent with proposals that suggest that 
conceptual digital design environments should 
emulate or parallel manual sketching (Bilda & 
Demirkan, 2003; Jonson, 2002). 

Summary 

The literature suggests that digital design 
tools that were developed specifically for con- 
ceptual design may provide substantive benefits 
over attempts to utilize conventional CAD in 
early design phases. In addition to ease of use 
and learning, stand-alone sketching programs 
provide the benefits of a nontraditional medium 
for design exploration supported by parallels 
with more traditional media. Consequently, this 
study was developed to investigate how a con- 
ceptual digital design application could be uti- 
lized by novice designers in an academic setting. 

Methodology 

As an exploratory study, the research 
methodology was structured to document stu- 
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dent perceptions of the effectiveness of concep- 
tual design software effectiveness in a design 
foundations course and also to investigate 
approaches to conducting the research and 
analysis. The study was structured to address 
the following research questions: 

1 . Does CAD experience influence the per- 
ceived effectiveness and use of the software? 

The null hypothesis for research question one 
was that prior CAD experience does not influ- 
ence the perceived effectiveness and use of the 
software. 

2. To what extent was the use of this soft- 
ware application perceived to influence the 
design process? The null hypothesis for 
research question two was that \the software was 
not perceived to influence the design process. 

3. What software tools/features/operations 
were perceived as effective and/or easy to use? 

Research Design 

The research design for this study was 
structured to utilize a mixed methodology; quan- 
titative data collected with a survey instrument 
was supported by supplemental data collection 
by the class instructor in the form of instructor 
observations and logs. It was the intention of the 
researchers that the instructor’s observations 
would be used to provide insights into the sur- 
vey responses and analysis of the quantitative 
data collected with the survey instrument. 

Population & Sample 

The study participants were students 
enrolled in the second course of the two-course 
architectural representation and design founda- 
tions sequence of the four-year Bachelor of 
Science program at a public university. This 
required class was offered once per academic 
year, open only to second-year architecture 
majors. The study population was a convenience 
sample; therefore, there was no attempt to ran- 
domly select participants from a larger group as 
the students were selected based on their enroll- 
ment in a specific course which would have 
required prior coursework involving manual 
drawings and physical models. This decision 
was based on the conclusion that students had to 
some extent developed skills with design con- 
cepts, composition, drafting, and model build- 
ing, which would provide a reference for their 
perception of the software effectiveness. 
Additionally, the course typically included a 


mixture of students with CAD skills and those 
without, therefore providing the potential for 
responding to the research question related to 
the influence of CAD experience. 

Instrumentation 

A survey was developed by the researchers 
to collect data related to the research questions. 
.Instructor’s observations were documented dur- 
ing the duration of the class activity. The survey 
utilized a series of Likert questions which docu- 
mented the participants’ experiences using the 
software in the study activities. However, it 
should be noted that the instrument was not pilot 
tested nor were reliability checks conducted for 
the Likert items. The questions were used to col- 
lect data on the participants’ CAD experience, 
and the influence of CAD skills in learning and 
using the software. The questions also were used 
to collect data on the participants’ perceived 
effectiveness of specific software features, tools, 
and operations, as well on the participants’ per- 
ceptions of the ease of use of various features 
and operations. Additional questions were used 
to collect data on the participants’ perceptions of 
the influence that this software had on their 
design process, and of the effectiveness of the 
software as a design and communication tool. 

The majority of the survey questions were 
based on a five-point rating scale. For questions 
intended to collect data on perceived effective- 
ness, the responses were “very effective” (5), 
“effective” (4), “neutral” (3), “ineffective” (2), 
and “very ineffective” (1). For questions intend- 
ed to collect data on perceived ease-of-use, the 
responses were “very easy” (5), “easy” (4), 
“neutral” (3), “difficult” (2), and “very difficult” 

(1) . Data was also collected for these categories 
with questions requiring a response of “strongly 
agree” (5), “agree” (4), “neutral” (3), “disagree” 

(2) , or “strongly disagree” (1). 

Statistical Analyses 

Statistical analyses were employed to 
answer the research questions for the study. The 
use of Likert-style questions limited the quanti- 
tative analysis to tests appropriate for ordinal 
data. To test for significant ratings above or 
below neutral, a single-sample t test with a test 
value of 3.00 was used to analyze the mean of 
the responses to each question. The confidence 
interval used for all tests and analysis was .95 (a 
=.05). 



Nonparametric correlation tests were used 
to analyze responses for all variables. Despite 
the inherent limitations associated with the use 
of nonparametric tests with the small size of the 
study population and assumption that the data 
was normally distributed, it was anticipated that 
cross-tab and correlation test could provide 
additional insight into the t-test results. 
Additionally, analysis using crosstab and corre- 
lation tests in the exploratory study could assist 
in evaluating the structure of the survey ques- 
tions and identifying minimum study population 
sizes required for further research. 

For analysis of CAD experience and skill 
level three primary variables were used. First, 
the participants were asked to identify the num- 
ber of CAD courses they had taken in both high 
school and college. This provided a measure that 
was to some degree objective. Second, they were 
asked to indicate their perception of their CAD 
proficiency using a Likert-item with a ranking 
of very proficient, proficient, moderate, mini- 
mal, or no experience. Lastly, the participants 
were also asked to indicate the extent to which 
they perceived their CAD skills to be useful in 
using the software on a Likert scale ranging 
from 1, not at all useful, to 5, very useful. 
Students with no prior CAD skills were exclud- 
ed from the statistical analysis. 

To determine the extent that the software 
was perceived to influence the design process, 
the survey included questions which asked the 
participants to rate the extent to which they per- 
ceived the use of the software had a positive 
effect on design outcomes and had a positive 
effect on design communication. Participants 
were also asked to rate the extent they perceived 
that the use of the software influenced their use 
of forms in the model and to rate the extent to 
which they believed they were able to create all 
the forms they intended to use in their model. 

To determine the perceived effectiveness of 
specific software features and operations, partic- 
ipants were asked to rate the effectiveness of the 
software relative to a variety of features and 
operations, including; 

1 . The use of the workspace and grid snap 

2. The use of object handles or grips to 
reshape or resize an object 

3. The use of object handles or grips to 
reposition an object 


4. The ability to add shadows to the display 

5. The ability to change to plan or elevation 
views 

The ratings ranged from very effective (5), 
to very ineffective (1), with a rating of 3 indicat- 
ing neutral. An additional response of “I did not 
use this feature” was also provided. These 
responses were excluded from the data analysis. 
Other operations were similarly rated for ease of 
use using a five-point ranking of five for “very 
easy” to one for “very difficult,” with a ranking 
of 3 indicating neutral. The operations were: 

1 . Creating rectangular/box-shaped volumes 

2. Creating curvilinear-shaped objects 

3. Creating objects with accurate and/or 
specific dimensions 

4. Using the grid tools to create objects at 
specific locations 

5. Using the grid tools to create objects 
with specific dimension 

6. Accurately re-positioning objects and 
forms in space 

7. Accurately placing/positioning objects 
and forms relative to other objects and/or 
forms 

8. Accurately placing/positioning objects at 
varying elevations or heights. 

Additionally, other operations were rated for 
perceived effectiveness using a five-point rank- 
ing of five for “strongly agree” that the opera- 
tion was effective to one for “strongly disagree” 
that the operation was effective, with a ranking 
of three indicating neutral. The operations were: 

1 . Interface/commands facilitated precise 
positioning/placement of objects. 

2. Interface/commands facilitated precise 
re-positioning/movement of objects. 

Data Collection Procedures 

The researchers utilized a design project in 
a second-year, first-semester design foundations 
course as the source for data collection related 
to the research questions. The project was 
selected because it had sufficient duration to 
enable the faculty to allocate time to introducing 
the software as part of the course activities. The 
project brief required students to design a struc- 
ture that “was enclosed but not enclosed” within 
a predefined context that included a pre-existing 
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slab and column grid located in a flat rectangu- 
lar open space surrounded on all four sides with 
dense trees. This was intended to provide the 
students with a pre-existing ordering system (the 
column grid) and a well-defined site perimeter 
(the wooded perimeter). The pre-existing com- 
ponents were to be integrated into their design 
(Figure 1). 

Initial class activities were directed towards 
exploring responses to the project brief and 
included manual sketching and development of 
physical models. As the students continued to 
use sketching and physical modeling, class time 
was allocated to introduce them to Autodesk’s 
Architectural Studio software, a conceptual 
design application that included a variety of 3- 
dimensional modeling tools. The software mod- 
eling tools included the capability to create 3D 
shapes using primitive objects such as cubes, 
spheres, and cylinders, and objects that could be 
created using free-form spline-based sketching 
tools. The software included a grid-snap feature 
(workspace grid) and the capability to easily 
create a user-defined grid on object-surfaces. 
The interface incorporated references to conven- 
tional drawing and drafting media and tech- 
niques. The drawing tools included both free- 
hand sketching emulation as well as basic 2D 
geometry commands used to create lines and 

Figure 1 . Study Project Handout 



shapes such as arcs and rectangles. This enabled 
students to utilize practices that had been pro- 
moted in class, including techniques such as 
overlaying layers of tracing paper and icon- 
based drawing tools with a functionality that at 
least to some extent paralleled pencil and felt 
markers used in the course. The use of interface 
elements that parallel conventional design repre- 
sentation techniques and processes is aligned 
with the parameters for digital conceptual design 
applications proposed by Bilda and Demirkan 
(2003). All students used identical computers 
configured with a mouse as the input device. 

Two software training sessions were con- 
ducted over the duration of the assignment. 
During and after the initial session, students 
were required to work with the software inde- 
pendently of the course project in order to pro- 
duce freeform compositions which incorporated 
both rectangular and curvilinear forms. After the 
second session, the students were asked to 
experiment by re-creating their course projects 
and developing them further using the software 
tools. For their final project submission, stu- 
dents were instructed they could utilize images 
generated from the computer model to supple- 
ment to their physical models and manually 
drafted documentation. 

The Survey was administered at the end of 
the primary design activities for the project, but 
prior to the student’s final presentation. This 
enabled students to have sufficient experience 
with the software to provide informed responses 
to the questions on software features as well as 
their overall experience in using the application 
as a design tool. Although the survey was 
administered during class time, students were 
informed that participation in the study was vol- 
untary. 

Supplemental data was collected using logs 
or journals of the instructors’ observations dur- 
ing training sessions and also during nonstruc- 
tured studio activities. These writings primarily 
focused on documenting commonly used fea- 
tures, commonly used command or software 
operation sequences, and aspects of software 
operations that students found to be more time- 
consuming or difficult. 

Findings 

Assumptions and Limitations 

Because of the small study population size, 
the extent to which the findings can be general- 


ized is inherently limited, and, since the study 
was structured around the operation and features 
of one software application, the generalization 
of conclusions to other applications is limited as 
well. It should be noted that any conclusions 
related to non parametric tests conducted on 
responses associated with prior CAD skills, an 
area of interest in this study, was further limited 
because students with no prior CAD skills were 
excluded from the statistical analysis, further 
reducing the sample size. 

The study population was also a conven- 
ience sample in that the students were selected 
in terms based on their status as novice design- 
ers in the architecture program at a specific 
institution, and their access to labs with the 
required software installed. Therefore, it could 
not be assured that this sample was an accurate 
representation of some larger group or popula- 
tion. Additionally, the use of Likert-scale ques- 
tions assumed that data was normally distributed 
and that all participants utilized a similar inter- 
pretation of the available categories and termi- 
nologies when responding to the questions. It 
was also assumed that there was only limited 
influence of the Hawthorne effect (Brannigan & 
Zwerman, 2001), which assumes that responses 
to the survey items were influenced by participa- 
tion in the study. Lastly, as the instructor’s 
observations were intended to be supplemental 
to the primary data collection using the survey 
instrument, these observations were assumed to 


Figure 2. Initial Experimentation (1st 
Session): Perspective and Plan View 



Figure 3. Software Experimentation 
(2nd Session) 



be anecdotal in the absence of a data-collection 
structure which supported more rigorous quali- 
tative analysis. 

Instructor Observations 

All participants in the study appeared to be 
able to navigate the menu selections and create 
basic geometry with little difficulty. The pri- 
mary modeling approach was to place objects, 
initially using only rectangular-shaped volumes, 
using the default predefined object height (10 
units) and then manipulate the geometry to the 
desired height, as well as length and width. In 
the initial explorations, students primarily uti- 
lized manipulations of height in the z direction 
with very limited manipulation of geometry off- 
axis relative to x, y, and z axis (Figure 2). 
Students appeared to utilize the ability to switch 
between orthographic and perspective views and 
also to utilize shadows and the ability to manip- 
ulate object transparency. Some students had 
difficulty with the zoom features when experi- 
menting with “field of view” options. Students 
would “get stuck” in perspective view after 
incorrectly performing zoom operations that 
produced a very wide field of view, resulting in 
a distorted on-screen view of their geometry. 

The students did not utilize software features 
that paralleled manual drafting, including the 
sketching tools such as multiple pens for varia- 
tions in line weight and style. Additionally, no 
students were observed using sketch overlays. 
During the second training session more experi- 
mentation with curved forms and off-axis geom- 
etry was observed (Figure 3). The use of the 
ground plane grid and grid-snap tools increased, 
as did attempts to use more precision in creating 
3D geometry, a factor largely associated with 
attempts to begin to use the software to build a 
computer model of their project. For the final 
project submission, most students included sup- 
plemental computer images when presenting 
their work. It was noted that nearly all submis- 
sions incorporated shadows and included plan 
or elevation views in the output (Figure 4). 

Data Analysis 

A total of 24 students were in the class, and 
23 participants completed the survey (N = 23). 
15 participants (65.2%) indicated they previous- 
ly had taken at least one CAD class. Of these, 
seven (30.4%) had taken one class, three had 
taken two classes, and one had taken three or 
more classes. The participants self-identified 
skill level was documented as follows: seven 
indicated that they had no CAD skills, four 
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Figure 4. Final Project submissions. 



indicated their CAD skills were minimal, 10 
indicated their CAD skills were moderate, and 
one indicated proficiency with CAD. These 
results indicate consistency between reported 
CAD skills and number of CAD courses. One 
participant did not respond to these two ques- 
tions, but did submit the survey and respond to 
the other questions on the survey. These respons- 
es are documented in Table 1 and Table 2. 

One sample t-tests yielded several statisti- 
cally significant results. T-test’s for the variables 
rating overall ease-of-use and overall ability to 
learn the software found a positive mean differ- 
ence of .64 (p = .002) and 1 .05 (p = .000) 
respectively. For specific tools and features, sta- 


tistically significant ratings were found for the 
effectiveness of workspace grid settings (p = 
.000), the effectiveness of handles or grip effec- 
tiveness to re-size objects (p =.000), the effec- 
tiveness of handles or grip to re-position objects 
(p = .000), the ability to rotate objects (p = 

.001), , the ability to add shadows (p = .000), the 
ability to change to plan or elevation views (p = 
.000), and creating rectangular volumes (p = 
.000). T-tests found that mean difference of the 
rating for the perception that the software influ- 
enced the forms used in the participants designs 
was also significant at the .01 level (p = .000). 
Additionally, no significant mean difference for 
the perception that the participants were able to 
create all the forms they needed (p = .576), 


Table 1 Respondent CAD Class Experience (N = 23) 


No prior 

One prior 

Two prior 

Three or more 

No 

CAD classes 

CAD class 

CAD classes 

prior CAD classes 

Response 

7 

11 

3 

1 

1 





Table 2 Respondent CAD skill self-assessment (N = 23) 


No CAD skills 

CAD skills: 

CAD skills: 

CAD skills: 

No 


Minimal 

Moderate 

Proficient 

Response 

7 

4 

10 

1 

1 


suggesting a largely neutral rating for this vari- 
able. Features and operations that generally 
related to accuracy and precision were found to 
be not significant or in some cases to have nega- 
tive mean differences, indicating an unfavorable 
rating. 

Although the small study population limited 
the interpretation of the results of nonparametric 
tests, the analysis were conducted in an attempt 
to identify areas in which both the study and the 
survey instrument could be improved. Cross-tab 
analysis of the responses yielded statistically 
significant relationships between the perceived 
usefulness of previous CAD skills and the ease- 
of-use of the software to modify objects to spe- 
cific dimensions (-.505, p = .002), and ease-of- 
use of the grid tools to create objects with spe- 
cific dimensions (-.359, p = .036). A significant 
relationship was found between perceived CAD 
skill level and ease-of-use of the software to cre- 
ate curved forms (.419, p = .035). Similarly, 
analysis using bivariate correlations (Kendall tau 
b) yielded a significant negative correlation 
between the perceived usefulness of previous 
CAD skills and the ease-of-use of the software 
to modify objects to specific dimensions (-.505, 
p = .016), and a positive correlation for ease-of- 
use of the software to create curved forms (.419, 
p = .047). Bi-variate correlation analysis yielded 
a significant negative correlation between num- 
ber of CAD classes and the ease of use of the 
software in placement and positioning objects (- 
.588, p = .014). 

Cross-tab analysis yielded statistically sig- 
nificant relationships between perceptions that 
the software had a positive effect on design out- 
comes and perceived overall software ease-of- 
use (.463, p = .006), the effectiveness of the 
software for facilitating students in creating 
forms and shapes needed for their designs (.518, 
p = .009), and the effectiveness of the software 
for communicating their design (.452, p = .030). 
Bivariate correlation analysis found statistically 
significant relationships between perceptions 
that the software was positive for communicat- 
ing their final project (.452, p = .024), and the 
effectiveness of the software tools and opera- 
tions for creating all the forms they intended for 


their design (.518, p = .010). When considering 
specific tools and features/operations, bivariate 
correlations were found between perceptions 
that the software had a positive effect on design 
outcomes and workspace and grid setting fea- 
tures (.523, p = .009), the effectiveness of 
object-handles or grips to re-size objects (.678, 
p = .001), the effectiveness of the interface and 
commands for precision in object 
placement/positioning (.435, p = .026), and the 
effectiveness of the interface and commands for 
precision in object relocation/re-positioning 
(.631, p = .001). 

Conclusions and Recommendations 

Analysis of the responses suggested for this 
group of students CAD skills, did not appear to 
have a positive influence on the use and opera- 
tion of the software, thus resulting in the null 
hypothesis for research question one not being 
rejected. It could be assumed that for these stu- 
dents’ experience using CAD software, which 
would typically include commands that 
employed a high degree of accuracy and preci- 
sion, was not easily translated to the modeling 
tasks using this type of application. Therefore, 
the data and observations in this exploratory 
study suggested that CAD skills may not be 
essential to the introduction of this type of appli- 
cation in this course and that the introduction 
earlier in the curriculum, at which time several 
students may have not had any CAD training or 
experience, may actually be favorable. Such a 
conclusion cannot be generalized to other stu- 
dents or curriculum because of the use of a con- 
venience sample and a small study population in 
this exploratory study. 

The responses suggested participants 
seemed to view the software positively influenc- 
ing the project design and communication, thus 
rejecting the null hypothesis for research ques- 
tion two. Even though it is necessary to consider 
the limitations of nonparametric tests in the con- 
text of this study, the positive correlations found 
between the perception that the software had a 
positive effect on design and the perceived abili- 
ty to create all forms needed for the design indi- 
cated this study population likely perceived the 
software most effective were those who were 
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most successful at developing the competencies 
they needed to model their projects. 

The students’ responses suggest that the 
software was to some extent perceived as easy to 
use and learn, as indicated by the ratings for 
these variables. However, these exploratory find- 
ings also suggested that on the whole the stu- 
dents were neutral in their rating of their ability 
to develop the required competencies; anthey 
also may have believed the software did influ- 
ence their designs. The data does suggest that 
the participants found certain tools and features 
more effective than others. For example, the sur- 
vey responses and the instructor’s observations 
suggested that the novice designers in this 
course with no CAD experience had the greatest 
difficulty in creating computer models with pre- 
cise dimensions and in creating curvilinear 
forms. Arguably, since creating curvilinear 
forms involves more complicated processes than 
creating rectilinear forms, it could be assumed 
that curved forms required greater technical 
competencies because this is the single variable 
where the positive influence of CAD skills was 
most evident. This result suggests that for inex- 
perienced students faculty may still find it nec- 
essary for allocation of time to learning activi- 
ties specifically associated with dimensional 
precision and more complex object modeling 
tasks are recommended in order to minimize the 
extent these features would be perceived as 
obstacles to design exploration. 

Inventory of recommended features 

The combination of the data sources, class- 
room observations, and student comments sug- 
gested that some 3D modeling tools and features 
were perceived to be either more effective or 
easier to use than others. Additionally, the extent 
of use of several tools and features also suggest- 
ed preferences among the participants.. 
Therefore, the study suggests that educators 
considering integrating digital modeling tools 
should consider applications that include the 
features described in the following paragraphs: 

Direct Geometry Manipulation 

The data indicated that the novice designers 
in the study preferred to directly manipulate the 
geometry, as evidenced by the statistically sig- 
nificant effectiveness ratings for the operation of 
handles and grips. This was supported by 
instructors’ observations, specifically in relation 
to the commonly observed technique of first 
creating an object and then modifying and 


manipulating the object to the desired dimension 
and location. 

Grid Tools 

The data also indicated that grid tools on 
both objects and workplace were utilized and 
considered effective. It is likely that the grid- 
snap tools provided a means by which students 
could create and align objects with precision 
without relying on typing in specific coordi- 
nates. However, it should be noted that although 
not statistically significant, correlation tests 
found negative correlation between perceived 
CAD skills, number of CAD classes, and the 
perceived usefulness of CAD skills and ratings 
for all questions related to effectiveness of the 
grid and workspace tools. Though the study did 
not document perceptions of the effectiveness 
and ease-of-use of coordinate entry, alternative 
methods of placement and manipulation may be 
considered an important feature. 

Orthographic Views 

It was observed that the ability to easily 
change between perspective and orthographic 
views was utilized extensively by the participants. 
This feature was also used for generating output 
at various stages during the assignment. One- 
sample t-tests found the mean difference of the 
rating for the effectiveness of the ability to 
change to orthographic views was significant at 
the .01 level (p = .000). The data also suggested 
an association between the perceived effective- 
ness of this feature and the ability to learn to use 
the software (.455, p = .006). While the combina- 
tion of observation, output, and data analysis sug- 
gested that this feature is effective, utilizing 
orthographic views in a digital design tool can 
also serve to make connections with related 
coursework and course activities that employ 
drafting techniques and other similar terminology. 

Shadow Display 

The data suggested that the display-shadow 
option was widely utilized in the assignment for 
output generated during the training sessions 
and for display of the participant’s solutions to 
the design problem. One-sample t-tests found 
the mean difference of the rating for the effec- 
tiveness of the ability to use shadows was signif- 
icant at the .01 level (p = .000). The perceived 
effectiveness of the ability to use shadows was 
significantly correlated with perceived ability to 
learn the software (.360. p = .025) and with the 
perceived effectiveness of the software in accu- 
rately placing objects at varying elevations 



(.555, p = .000). The study did not yield any data 
that suggested an explanation and no assumptions 
can be directly inferred from the documentation. 
However, given the extensive use of shadows 
observed suggested that this feature was utilized 
to an extent that it may be an important option in 
any application used in a design course. 

Survey Instrument and Methodology 
Modifications 

The results of this exploratory study suggest 
several adjustments in methodology and in the 
survey instrument. In future studies a minimum 
study population size of approximately 48 would 
be more appropriate in order to assure minimum 
cell counts required for many nonparametric tests. 
Along with the increased population size, analysis 
should also include distribution tests to determine 
the extent to which assumptions of normal distri- 
butions can be made to more effectively support 
any interpretations and conclusions. Analysis of 
responses that used t-tests should also be expand- 
ed. Since a one-sample median test allows for 
testing to determine if a sample median differs 
significantly from a hypothesized value, future 
studies should consider median t-tests in addition 
to mean t-tests in order to more effectively sup- 
port conclusions. 

Several recommendations for modifications 
to the survey instrument arose from the analysis 
related to the nonparametric statistical analysis 
used in this study. Most of these modifications 
were related to the size of the study population. 
For nonparametric tests, restructuring the Likert 
questions from five items to three would more 
likely yield tests results that met the minimum 
number of responses required in future studies 
with small populations. Additionally, the instru- 
ment used in this exploratory study was struc- 
tured to collect data using t-tests and nonparamet- 
ric tests. Questions should be restructured to col- 
lect data that can be more clearly associated with 
a specific statistical test associated with a specific 
research question rather than attempting to the 
somewhat uniform approach to all research ques- 
tions and tests that was employed in this study. 
Lastly, since the role of CAD experience was an 
area of interest in this study, in future studies the 
instrument should be modified to collect addi- 
tional demographic data related to prior CAD 
experience in order to provide a more thorough 
description of the participants’ skill sets. In larger 
studies, this change could assist in identifying 
similarities, differences, and subcategories among 


the students with CAD experience in order to 
draw more informed conclusions related to com- 
parisons between students with CAD experience 
and those without. 

Areas for Further Research 

The study findings suggest that digital design 
tools can be integrated in such courses without 
CAD skills or related technical knowledge as a 
prerequisite, and that these applications can serve 
to facilitate design exploration and communica- 
tion for novice designers. However, additional 
empirical research into the perceived effective- 
ness of conceptual design software in design 
foundations courses is necessary to substantiate 
the findings of this exploratory study. The initial 
analysis provides insight into the influence of 
some of the variables that could affect student 
performance and success in meeting intended stu- 
dio learning outcomes. Specifically, investigations 
with larger populations that compare alternative 
software applications could assist faculty in iden- 
tifying the features in common among the appli- 
cations that proved to be perceived as most effec- 
tive by users. Expanding the investigation to 
include multiple disciplines may provide insight 
into the fields and areas of study in which these 
applications held potential for enhancing out- 
comes. Additionally, given the data and the 
instructors’ observations, the findings related to 
use of features such as direct geometry manipula- 
tion suggest investigations of the role of alterna- 
tive input devices may be appropriate as well in 
order to identify ways faculty could optimize dig- 
ital conceptual design tools in academic environ- 
ments. 
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